[BREAKING CHANGES] Move agreement negotiation off-chain, keep only mutual commitment on-chain#105
Open
danielbui12 wants to merge 53 commits into
Open
[BREAKING CHANGES] Move agreement negotiation off-chain, keep only mutual commitment on-chain#105danielbui12 wants to merge 53 commits into
danielbui12 wants to merge 53 commits into
Conversation
`create_bucket_with_storage` takes an explicit provider account; then the pallet then performs an O(1) lookup of that single provider and re-validates all constraints before opening the agreement
Adding new AgreementTerms type for data signing Adding ReplayWindow and ProviderReplayState to prevent signature replay
Introduce a single-call flow where a provider signs storage terms off-chain and the owner redeems them on-chain, replacing the request/accept dance for primary agreements.
Bucket creation and agreement opening are folded into one atomic extrinsic.
Primitives (`storage-primitives`):
- `AgreementTerms<AccountId, Balance, BlockNumber>`: provider-signed quote carrying owner, max_bytes, duration, price_per_byte, valid_until, nonce.
- `ReplayWindow`: per-provider 256-slot sliding window over signed nonces (`hwm` + 32-byte bitmap, LSB = hwm). `try_accept(nonce)` shifts the bitmap on forward jumps and rejects duplicates (`AlreadyUsed`) or out-of-window pasts (`TooOld`). Covered by 7 unit tests including out-of-order, edge, and large-jump cases.
Pallet (`pallet-storage-provider`):
- `ProviderReplayState` storage map (`AccountId -> ReplayWindow`).
- `establish_storage_agreement` extrinsic + pub `establish_storage_agreement_internal(owner, provider, terms, sig)` helper for Layer 1 reuse. Verifies `MultiSignature` over `blake2_256(SCALE(terms))`, checks `valid_until`, advances the replay window, then runs the existing provider/capacity/stake/duration/price validation before creating the bucket + primary agreement.
- New errors: `InvalidProviderSignature`, `TermsExpired`, `NonceAlreadyUsed`, `NonceTooOld`, `TermsOwnerMismatch`.
- New event: `StorageAgreementEstablished { bucket_id, provider, owner, terms, expires_at }`. Named `Storage*` so a future `establish_replica_sync_agreement` flow can sit alongside.
The legacy `create_bucket_with_storage` extrinsic is left in place for now; it will be removed in a follow-up once callers migrate.
Refactor replica agreement creation to use the same provider-signed terms flow as establish_storage_agreement, eliminating the pending request/accept stage. - recover `replica_params` in `AgreementTerms` - migrate `request_agreement` extrinsic to `establish_replica_agreement`: redeems provider-signed terms against an existing bucket, verifying signature and replay window before opening the agreement atomically. - migrate `request_replica_agreement_internal` to `establish_replica_agreement_internal`: mirrors `establish_storage_agreement_internal` for higher-layer pallets. - Drop the `AgreementRequest` storage and struct, the cleanup_bucket drain loop, the `AgreementRequested`/`AgreementRejected`/`AgreementRequestWithdrawn` events, and the now-unused request-related errors. - Add `ReplicaAgreementEstablished` event and `MissingReplicaTerms` error. The pallets in storage-interfaces/, benchmarks, tests, and client SDK still reference the old names and need a follow-up pass.
Update pallet/src/tests.rs to exercise the establish_storage_agreement / establish_replica_agreement extrinsics that replaced the legacy request/accept flow, and drop create_bucket / create_bucket_with_storage which are no longer exist. Test helpers: - Add sr25519 signing helpers (generate_provider_public_key, sign_terms) that use the runtime keystore registered in mock.rs. - Add primary_terms / replica_terms builders and a register_signing_provider helper for the common setup. And more test cases for new extrinsics & changes.
- Remove benchmarks for deleted extrinsics. - Add establish_storage_agreement and establish_replica_agreement benchmarks covering signature verification + replay-window mutation + bucket / agreement insertion costs. - Update helper functions, other benchmarks regarding new changes.
- Update create_s3_bucket now takes (name, provider, terms, sig) and calls establish_storage_agreement_internal for the Layer 0 bucket + primary agreement atomically. - `create_s3_bucket_with_storage` is removed. - Drop NoProvidersAvailable, AgreementRequestFailed, Layer0BucketCreationFailed errors, and return Layer 0 errors directly. - Update tests following changes
pallet-drive-registry - create_drive is updated following new flow. - allocate_bucket_for_user is removed. - Remove unnecessary code, update tests and benchmarks runtime - drop genesis bucket on Layer 0
- Add client/src/agreement.rs with the AgreementTermsOf mirror type, NegotiateRequest / SignedTerms wire shapes, a hex-bytes MultiSignature serde adapter, and a sign_terms helper that matches the on-chain blake2_256(SCALE(terms)) verification. - AdminClient: replace create_bucket + request_agreement + withdraw_agreement_request + terminate-style request/accept helpers with establish_storage_agreement(provider, terms, sig), which parses the new BucketCreated event to surface the bucket id. - ProviderClient: drop accept_agreement / list_pending_requests / reject_agreement_request; add mock negotiate_terms HTTP client that POSTs to a provider node `/negotiate` endpoint and returns SignedTerms. - Update complete_workflow.rs, and tests to the new flow.
…-node - Add provider-node/src/negotiate.rs: - NonceCounter: atomic monotonic counter persisted to disk on every allocation, can continue with on-chain hwm. - sign_terms() mirrors the on-chain verifier: blake2_256(SCALE(terms)) → sr25519 sign → MultiSignature::Sr25519. - Wire POST negotiate in api.rs: allocates the next nonce, builds AgreementTerms, signs, then returns SignedTerms (error 503 if the node has no signing key). - command.rs: drop start_agreement_coordinator; replace with setup_nonce_counter. - Delete provider-node/src/agreement_coordinator.rs.
Renames the ReplayWindow anchor field and all its callers from `hwm` (high-water mark) to `hsn` (high sequence nonce), which more accurately describes that it tracks the highest accepted agreement-term nonce: - ReplayWindow.hwm -> hsn (+ doc/comment updates) - ProviderClient::fetch_replay_hwm -> fetch_replay_hsn - NonceCounter::bootstrap_from_hwm -> bootstrap_from_hsn - setup_nonce_counter starts the counter at new(1), dropping the redundant bootstrap_from_hwm(0) Pure rename + identifier change; no behavioral or SCALE-encoding change.
…s flow The pallets replaced open-ended bucket creation (create_bucket, create_bucket_with_storage, request_primary_agreement) with the negotiate-then-redeem flow, so the precompiles follow: - storage-provider: drop createBucket/createBucketWithStorage/ requestPrimaryAgreement; add establishStorageAgreement(provider, terms, signature) returning the new bucket id - s3-registry: createS3Bucket now redeems provider-signed terms; drop createS3BucketWithStorage - drive-registry: createDrive now takes (name, provider, terms, signature) AgreementTerms/ReplicaTerms cross the Solidity boundary as PrimitiveAgreementTerms/PrimitiveReplicaTerms structs declared in each interface (alloy::sol! cannot resolve Solidity imports, so the mirrors are per-file copies); the signature is the SCALE-encoded MultiSignature from the provider's /negotiate response. Example contracts and interface copies updated to match.
mudigal
requested changes
Jun 3, 2026
Collaborator
mudigal
left a comment
There was a problem hiding this comment.
Good one overall - Some changes are needed to avoid certain attacks.
* feat: updating conosle-ui
* chore: double check runtime side
* fix(console-ui): send MultiSignature payload as 0x-hex for PAPI v2 isCompat
`buildSignedTermsArgs` was passing the 64-byte sig payload to `Enum()` as
a raw `Uint8Array`. PAPI v2's `isCompatible` rejects raw bytes for
fixed-length binary fields (`SizedHex<N>`) — its check is
`typeof value === "string" && value.startsWith("0x")` — and throws
`Incompatible runtime entry Tx(S3Registry.create_s3_bucket)` before the
tx is encoded. Variable-length binary (`Vec<u8>`) still wants a
`Uint8Array`; only fixed-length wants the hex string.
Encode the sig payload as a `0x`-prefixed hex string so it matches the
descriptor type cli 0.21.x generates for MultiSignature variants.
* fix(ui): set allowBuilds.esbuild=true for pnpm 11 install in subprocesses
pnpm 11.1.2 treats a placeholder/missing build-script approval as a hard
`[ERR_PNPM_IGNORED_BUILDS]` error when run from a non-TTY subprocess
(vite's `runDepsStatusCheck` spawns `pnpm install` this way). The
checked-in placeholder `esbuild: set this to true or false` is a literal
string, not a boolean, so pnpm 11 errored out and vite refused to start.
Replace the placeholder + `ignoredBuiltDependencies` entry with the
actual `allowBuilds: { esbuild: true }` — esbuild's postinstall just
links the platform binary, which we want to run anyway.
* test(console-ui): drive e2e bucket creation through the real UI flow
- Add createBucketViaUi / createBucketInFreshContext helpers that
fill the form, click "Choose Provider & Create", then pick the first
provider in the picker — the same path a real user walks.
- Use the UI flow from bucket-create, encryption, members, and
s3-objects specs instead of the chain-side createBucketViaApi shortcut.
- Add provider-picker / provider-picker-select testids on
ProviderPickerDialog so the picker is addressable.
- Rewrite createBucketViaApi + createDriveViaApi in test-helpers to do
HTTP /negotiate + atomic create_s3_bucket / create_drive, matching
the new on-chain shape (provider, terms, sig).
* feat(drive-ui): rewire create-drive for the negotiate → atomic establish flow
drive-client:
- Add negotiateTerms / buildSignedTermsArgs / listAvailableProviders.
MultiSignature inner is hex string (SizedBytes(64) is Codec<string>
in PAPI v2).
- Replace createDrive(options) with submitCreateDrive(name, provider,
providerUrl, signed) — only the chain step, takes pre-negotiated terms
so a failed submit can retry without re-negotiating.
- Drop the obsolete waitForProvider poll (atomic flow → primary_providers
is populated synchronously) and the `payment` field from DriveInfo.
state hook:
- createDrive(input) orchestrates negotiate → submit explicitly. Stash
retry context per creation so retryCreation(id) re-fires just the
chain step.
- Narrow CreationStage to submitting | ready | failed (drop the now-dead
created / waiting stages).
- Expose listAvailableProviders for the picker.
NewDriveDialog + ProviderPickerPanel:
- Embed the provider picker inline in the create dialog (no separate
modal). Picking a provider IS the submit; drop the "Choose Provider &
Create" button. Form drops payment / minProviders, adds pricePerByte.
- Status card adds a Retry on-chain submit button for failures after a
successful negotiate. "Unlimited" rendered when maxCapacity == 0n.
E2E:
- New helpers createDriveViaUi(page, name) and createDriveInFreshContext(
browser, name) that drive the form + embedded picker.
- members / persistence / file-ops / realtime specs replace
createDriveViaApi setup with the UI-driven helpers. Drop stale
payment / minProviders / commitStrategy props.
- drive-create.spec.ts walks the embedded picker (no submit button click).
* chore(provider-ui): disable Agreements page pending flow rework
Comment out the /agreements route, nav entry, and matching e2e test
while the agreement request flow is being reworked. Also switch
expected block time to Aura.SlotDuration.
---------
Co-authored-by: Ilia Churin <ilia@parity.io>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
The /info endpoint now reports the signer's real SS58 account instead of a placeholder, so the integration test derives the expected id from the //Alice seed rather than asserting "0xtest_provider".
…face Re-run benchmarks for pallet_storage_provider, pallet_drive_registry and pallet_s3_registry on both runtimes after bucket/drive creation moved to the provider-signed terms flow.
The precompiles' bucket/drive creation selectors now redeem provider-signed AgreementTerms, so the PAPI demos follow: - sc-api.js: add h160ToSubstrate (AccountId32Mapper fallback account for unmapped H160s, i.e. deployed contracts) and negotiatePrecompileTerms (POST /negotiate shaped for the PrimitiveAgreementTerms ABI struct) - sc-coverage.js: createBucket/createBucketWithStorage and requestPrimaryAgreement + accept_agreement replaced by establishStorageAgreement; createDrive takes (name, provider, terms, sig); renumbered to the 13 remaining selectors - sc-flow.js / sc-team-drive.js / sc-token-gated.js: negotiate terms after deploy with the contract's substrate-mapped account as terms.owner (the contract is the precompile caller), then pass the signed bundle through buyStorage / createTeam / initialize
Bumping REPLAY_WINDOW_BITS to 1024 grows the bitmap past the 32-element derive limit, so Default and serde get manual impls. Also fixes a hardcoded 32-byte bound in shift_left_le left over from the 256-bit window, derives BIT_MAP_WINDOW_SIZE from REPLAY_WINDOW_BITS, and updates tests whose anchors/expectations assumed the old width.
Add `bucket_id: Option<BucketId>` to `AgreementTerms` so the provider's signed quote is bound to the bucket it targets: - None for primary terms — the bucket is created at redemption; `establish_storage_agreement_internal` rejects bucket-bound terms. - Some(id) for replica terms — `establish_replica_agreement_internal` requires it to match the extrinsic's `bucket_id` (TermsBucketMismatch). Mirror the field as `hasBucketId`/`bucketId` in the precompile Solidity interfaces and decode it in `decode_terms`; guard the example contracts' primary entry points; map it through the PAPI demos' negotiate helpers; update the provider node's /negotiate, the client SDK wire types, and the UIs' terms handling accordingly. Also use `CheckMetadataHash::new(false)` in the paseo runtime tests to match the runtime's eth path — `new(true)` requires the metadata-hash build env, which tests don't set.
The /negotiate handler blindly signed whatever terms the client proposed (including price_per_byte=0) and the on-chain extrinsic trusts that signature as provider consent. It was also an unauthenticated nonce/CPU burner. - Fetch the provider's on-chain registration info at startup (ProviderClient::get_provider_info) and store it in ProviderState; expose it via /info - Reject terms below the listed price, outside duration bounds, beyond remaining capacity, or against closed acceptance flags - before a nonce is allocated or anything is signed - Add typed rejection errors (422) plus provider_info_unavailable (503) and rate_limited (429) - Rate-limit /negotiate (5 req/s, burst 16) via tower RateLimit+Buffer - Enable checkpoint coordinator in just start-provider and CI
… missing - /negotiate now returns SigningUnavailable (503) instead of a generic 500 when the node has no keypair or nonce counter - make nonce counter and provider info optional at startup instead of failing or silently starting from a default nonce - initialize/remove ProviderReplayStates on provider (de)registration
The signed payload is now blake2_256(TERM_CONTEXT | SCALE(terms)), with PRIMARY_TERM_CONTEXT = 'primary-term-v1:' and REPLICA_TERM_CONTEXT = 'replica-term-v1:'. The verifier takes the context from the redemption path rather than the terms themselves, so a quote signed for one flavour can never be redeemed as the other.
Pass max_bytes and price_per_byte as BigInt in all negotiateTerms call sites, matching the PAPI descriptor types. Raw JSON numbers fail on the provider's u128 fields because serde's untagged enum buffers through a Content type that cannot represent u128; the existing JSON.stringify replacer serializes BigInt as strings, which parse via FromStr.
The provider node now requires its account to be registered on chain at startup (setup_provider_info fails hard otherwise), so the demos can no longer be the ones to register Alice/Charlie after the nodes are up. Register both providers via the register_provider example right after the parachain produces blocks, and give each provider its own log file so the disk node no longer clobbers the inmemory node's log.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Primary Changes
pallet-storage-provider(Layer 0)establish_storage_agreement(provider, terms, sig)extrinsic — atomically verifies provider sig, checks replay window + expiry, creates bucket, opens agreement for primary provider.AgreementTerms (owner, max_bytes, duration, price_per_byte, valid_until, nonce),ProviderReplayStates (hwm + 256-bit bitmap per provider).MultiSignature::verify(blake2_256(SCALE(terms)))against the provider's registered public key.establish_storage_agreement_internalreused bypallet-s3-registry&pallet-drive-registry.storage + related events/errors.
Layer 1
pallet-s3-registry::create_s3_bucketandpallet-drive-registry::create_drivenow take (terms, sig) and call topallet-storage-provider::establish_storage_agreement_internal; removed old & unused code.client/ (Rust SDK)
AgreementTermsOf, andsign_terms()helper.AdminClient::establish_storage_agreement(provider, terms, sig)replaces old flow.ProviderClient::negotiate_terms(url, req)invokes POST/negotiatefromprovider-nodeto negotiate and get provider's signature on agreements.provider-node/
/negotiate→ provider allocates nonce (per request), signs terms, returnsSignedTerms.NonceCounterthat bootstraps from chainhwmon cold start.NonceCounteris used to prevent signature replay attack.Issues
Follow-up issue: